How Can We Argue With Performance Indicators?

We seem to be living in an age when the only way we can justify an organisational activity is with the use of numbers. Numbers, on the other hand, appear unarguable. The only way to argue with numbers is to find different numbers. To us numbers carry the imprimatur of truth. Yet, in 1979, Campbell documented his pessimistic laws of quantitative indicators. The more any quantitative indicator is used for decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social process it is intended to monitor (p 85). Most recent analyses of the dysfunctional effects of performance indicators have focused on the content of the performance indicators. These analyses emphasise the sorts of responses that occur when the values and theories inherent in performance indicators conflict with those of people associated with the program. Is it possible then, to fill performance indicators with a new content and use them in the service of different values? Or do performance indicators, by their very nature, tend to promote particular value and theoretical positions? In this thesis I use the discourse analysis techniques of Foucault and Ellul to argue that those same factors that make performance indicators so powerful and so unarguable, also make it difficult to avoid their dysfunctional effects. The dysfunctional effects seem to be inherent in the discourse from which performance indicators gain their power, a discourse which justifies, and make possible, both modern approaches to production and the unrestrained development of technology. (Ellul calls this imperative progress of technology, “la technique”.) The act of freeing performance indicators from their dysfunctional effects will inevitably destroy their main advantage, certainty the certainty that allows action the certainty that comes from a particular abridged form of thought that is dominant in our society; thought that depends on ‘seeing’ rather than reasoning. The key arguments of the thesis are diagrammed in figure 1 (p 18). Foucault uses Bentham’s “panopticon” to demonstrate the way in which power operates in our society, of which performance indicators are a supreme example. The panopticon is a system of surveillance that turns those who are observed into their own, most vigilant, observers. As a system of power in the social, economic and political realms this produces compliance. However, as a system of knowledge production is produces reactivity. This mode of knowing has become the dominant determinant, not only of how we gain knowledge about the world, but also of how we come to know ourselves, (with severe consequences). Its purpose is to cause us to use our bodies productively and therefore the only aspect of “truth” with which it is concerned is ‘product’ or ‘effectiveness’. This is a severe form of reductionism. If we stay within this framework performance indicators are unarguable. If we move outside it, they become almost trivial. This certainty and reduction are achieved through definitional operationism. The fundamental fallacy of performance indicators, (and the cause of many dysfunctional effects), is the assumption that the relationship between an indicator and what it is trying to measure, is constant. Many modern decision-makers would simply not know how to act without the certainty that this simplifying assumption brings. Thus the indicator (which I shall call a symbol) and its target diverge, the symbol is likely to be retained while the underlying goal disappears. Indicators can end up becoming more real than that which they are intended to measure. We can easily end up living in a world of symbols whose real world content has long since been forgotten. This analysis is informed by a case-study of the attempt to develop a non-damaging, performance indicator system in a private rehabilitation hospital for people with acquired brain damage (ABD). This case study demonstrates the subtle way in which the reductionism inherent in performance indicators could end up undermining the program. I conclude by proposing that we need to abandon our ideal of performance indicators as a form of ‘seeing’ and to learn to think in more complex terms. I suggest that the use of program theory, as an heuristic technique, can help us do justice to the true complexity of programs while still allowing us to act and make decisions.